130 research outputs found
Deep Expander Networks: Efficient Deep Networks from Graph Theory
Efficient CNN designs like ResNets and DenseNet were proposed to improve
accuracy vs efficiency trade-offs. They essentially increased the connectivity,
allowing efficient information flow across layers. Inspired by these
techniques, we propose to model connections between filters of a CNN using
graphs which are simultaneously sparse and well connected. Sparsity results in
efficiency while well connectedness can preserve the expressive power of the
CNNs. We use a well-studied class of graphs from theoretical computer science
that satisfies these properties known as Expander graphs. Expander graphs are
used to model connections between filters in CNNs to design networks called
X-Nets. We present two guarantees on the connectivity of X-Nets: Each node
influences every node in a layer in logarithmic steps, and the number of paths
between two sets of nodes is proportional to the product of their sizes. We
also propose efficient training and inference algorithms, making it possible to
train deeper and wider X-Nets effectively.
Expander based models give a 4% improvement in accuracy on MobileNet over
grouped convolutions, a popular technique, which has the same sparsity but
worse connectivity. X-Nets give better performance trade-offs than the original
ResNet and DenseNet-BC architectures. We achieve model sizes comparable to
state-of-the-art pruning techniques using our simple architecture design,
without any pruning. We hope that this work motivates other approaches to
utilize results from graph theory to develop efficient network architectures.Comment: ECCV'1
Random Unitaries Give Quantum Expanders
We show that randomly choosing the matrices in a completely positive map from
the unitary group gives a quantum expander. We consider Hermitian and
non-Hermitian cases, and we provide asymptotically tight bounds in the
Hermitian case on the typical value of the second largest eigenvalue. The key
idea is the use of Schwinger-Dyson equations from lattice gauge theory to
efficiently compute averages over the unitary group.Comment: 14 pages, 1 figur
Overlap properties of geometric expanders
The {\em overlap number} of a finite -uniform hypergraph is
defined as the largest constant such that no matter how we map
the vertices of into , there is a point covered by at least a
-fraction of the simplices induced by the images of its hyperedges.
In~\cite{Gro2}, motivated by the search for an analogue of the notion of graph
expansion for higher dimensional simplicial complexes, it was asked whether or
not there exists a sequence of arbitrarily large
-uniform hypergraphs with bounded degree, for which . Using both random methods and explicit constructions, we answer this
question positively by constructing infinite families of -uniform
hypergraphs with bounded degree such that their overlap numbers are bounded
from below by a positive constant . We also show that, for every ,
the best value of the constant that can be achieved by such a
construction is asymptotically equal to the limit of the overlap numbers of the
complete -uniform hypergraphs with vertices, as
. For the proof of the latter statement, we establish the
following geometric partitioning result of independent interest. For any
and any , there exists satisfying the
following condition. For any , for any point and
for any finite Borel measure on with respect to which
every hyperplane has measure , there is a partition into measurable parts of equal measure such that all but
at most an -fraction of the -tuples
have the property that either all simplices with
one vertex in each contain or none of these simplices contain
MV3: A new word based stream cipher using rapid mixing and revolving buffers
MV3 is a new word based stream cipher for encrypting long streams of data. A
direct adaptation of a byte based cipher such as RC4 into a 32- or 64-bit word
version will obviously need vast amounts of memory. This scaling issue
necessitates a look for new components and principles, as well as mathematical
analysis to justify their use. Our approach, like RC4's, is based on rapidly
mixing random walks on directed graphs (that is, walks which reach a random
state quickly, from any starting point). We begin with some well understood
walks, and then introduce nonlinearity in their steps in order to improve
security and show long term statistical correlations are negligible. To
minimize the short term correlations, as well as to deter attacks using
equations involving successive outputs, we provide a method for sequencing the
outputs derived from the walk using three revolving buffers. The cipher is fast
-- it runs at a speed of less than 5 cycles per byte on a Pentium IV processor.
A word based cipher needs to output more bits per step, which exposes more
correlations for attacks. Moreover we seek simplicity of construction and
transparent analysis. To meet these requirements, we use a larger state and
claim security corresponding to only a fraction of it. Our design is for an
adequately secure word-based cipher; our very preliminary estimate puts the
security close to exhaustive search for keys of size < 256 bits.Comment: 27 pages, shortened version will appear in "Topics in Cryptology -
CT-RSA 2007
The spectrum of the random environment and localization of noise
We consider random walk on a mildly random environment on finite transitive
d- regular graphs of increasing girth. After scaling and centering, the
analytic spectrum of the transition matrix converges in distribution to a
Gaussian noise. An interesting phenomenon occurs at d = 2: as the limit graph
changes from a regular tree to the integers, the noise becomes localized.Comment: 18 pages, 1 figur
Comparing the reliability of networks by spectral analysis
We provide a method for the ranking of the reliability of two networks with
the same connectance. Our method is based on the Cheeger constant linking the
topological property of a network with its spectrum. We first analyze a set of
twisted rings with the same connectance and degree distribution, and obtain the
ranking of their reliability using their eigenvalue gaps. The results are
generalized to general networks using the method of rewiring. The success of
our ranking method is verified numerically for the IEEE57, the
Erd\H{o}s-R\'enyi, and the Small-World networks.Comment: 7 pages, 3 figure
Linear-time list recovery of high-rate expander codes
We show that expander codes, when properly instantiated, are high-rate list
recoverable codes with linear-time list recovery algorithms. List recoverable
codes have been useful recently in constructing efficiently list-decodable
codes, as well as explicit constructions of matrices for compressive sensing
and group testing. Previous list recoverable codes with linear-time decoding
algorithms have all had rate at most 1/2; in contrast, our codes can have rate
for any . We can plug our high-rate codes into a
construction of Meir (2014) to obtain linear-time list recoverable codes of
arbitrary rates, which approach the optimal trade-off between the number of
non-trivial lists provided and the rate of the code. While list-recovery is
interesting on its own, our primary motivation is applications to
list-decoding. A slight strengthening of our result would implies linear-time
and optimally list-decodable codes for all rates, and our work is a step in the
direction of solving this important problem
Majority Dynamics and Aggregation of Information in Social Networks
Consider n individuals who, by popular vote, choose among q >= 2
alternatives, one of which is "better" than the others. Assume that each
individual votes independently at random, and that the probability of voting
for the better alternative is larger than the probability of voting for any
other. It follows from the law of large numbers that a plurality vote among the
n individuals would result in the correct outcome, with probability approaching
one exponentially quickly as n tends to infinity. Our interest in this paper is
in a variant of the process above where, after forming their initial opinions,
the voters update their decisions based on some interaction with their
neighbors in a social network. Our main example is "majority dynamics", in
which each voter adopts the most popular opinion among its friends. The
interaction repeats for some number of rounds and is then followed by a
population-wide plurality vote.
The question we tackle is that of "efficient aggregation of information": in
which cases is the better alternative chosen with probability approaching one
as n tends to infinity? Conversely, for which sequences of growing graphs does
aggregation fail, so that the wrong alternative gets chosen with probability
bounded away from zero? We construct a family of examples in which interaction
prevents efficient aggregation of information, and give a condition on the
social network which ensures that aggregation occurs. For the case of majority
dynamics we also investigate the question of unanimity in the limit. In
particular, if the voters' social network is an expander graph, we show that if
the initial population is sufficiently biased towards a particular alternative
then that alternative will eventually become the unanimous preference of the
entire population.Comment: 22 page
Expansion in SL_d(Z/qZ), q arbitrary
Let S be a fixed finite symmetric subset of SL_d(Z), and assume that it
generates a Zariski-dense subgroup G. We show that the Cayley graphs of pi_q(G)
with respect to the generating set pi_q(S) form a family of expanders, where
pi_q is the projection map Z->Z/qZ
- …